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Abstract 


The  Carnegie  Mellon  Software  Engineering  Institute  held  the  U.S.  Army  Product  Line  Practice 
Workshop  on  February  12,  2009.  The  workshop  was  a  hands-on  meeting  to  share  Army  and  DoD 
product  line  practices,  experiences,  and  issues  and  to  discuss  specific  product  line  practices  and 
operational  accomplishments.  Participants  reported  encouraging  progress  on  Army  software 
product  lines.  This  report  synthesizes  the  workshop  presentations  and  discussions. 
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1  Introduction 


1.1  Product  Line  Practice 

A  software  product  line  is  a  set  of  software- intensive  systems  sharing  a  common,  managed  set  of 
features  that  satisfy  the  specific  needs  of  a  particular  market  segment  or  mission  and  that  are  de¬ 
veloped  from  a  common  set  of  core  assets  in  a  prescribed  way  [Clements  2002].  An  increasing 
number  of  organizations  are  building  their  products  as  product  lines  in  order  to  achieve  large- 
scale  productivity  gains,  improve  time  to  field  or  market,  maintain  a  market  presence,  compensate 
for  an  inability  to  hire,  leverage  existing  resources,  and  achieve  mass  customization. 

In  January  1997,  the  Carnegie  Mellon  ’'  Software  Engineering  Institute  (SEI)  launched  the  Product 
Line  Practice  Initiative  to  help  facilitate  and  accelerate  the  transition  to  sound  software  engineer¬ 
ing  practices  using  a  product  line  approach.  The  goal  of  this  initiative  is  to  provide  organizations 
with  an  integrated  business  and  technical  approach  to  systematic  reuse,  so  they  can  produce  and 
maintain  similar  systems  of  predictable  quality  more  efficiently  and  at  a  lower  cost. 

A  key  strategy  for  achieving  this  goal  has  been  the  creation  of  a  framework  for  product  line  prac¬ 
tice.  The  SEI  Framework  for  Software  Product  Line  PracticeSM  (henceforth  referred  to  as  “the 
framework”)  describes  the  foundational  product  line  concepts  and  identifies  the  essential  activities 
and  practices  that  an  organization  must  master  before  it  can  expect  to  successfully  field  a  product 
line  of  software  or  software- intensive  systems.  The  framework  is  a  living  document  that  is  evolv¬ 
ing  as  experience  with  product  line  practice  grows.  Version  4.0  is  described  in  the  book  Software 
Product  Lines:  Practices  and  Patterns  [Clements  2002],  and  the  latest  version  is  available  on  the 
SEI  website  [Northrop  2009b], 

The  framework’s  contents  are  based  on  information-gathering  workshops,* 1  extensive  work  with 
collaboration  partners,  surveys  and  investigations,  and  continued  research.  The  SEI  has  also  in¬ 
corporated  practices  reported  at  its  international  Software  Product  Line  Conferences  and  collected 
from  the  community  [Donohoe  2000,  Chastek  2002a,  Nord  2004,  Obbink  2005,  O’Brien  2006, 
IEEE  2007,  Geppert  2008], 

In  March  1 998,  the  SEI  hosted  its  first  Department  of  Defense  (DoD)  product  line  practice  work¬ 
shop,  Product  Lines:  Bridging  the  Gap — Commercial  Success  to  DoD  Practice  [Bergey  1998]. 
Topics  discussed  and  documented  included  DoD  barriers  and  mitigation  strategies,  and  similari¬ 
ties  and  differences  between  DoD  product  line  practice  and  commercial  product  line  practices. 


®  Carnegie  Mellon  is  registered  in  the  U.S.  Patent  and  Trademark  Office  by  Carnegie  Mellon  University. 

SM  Framework  for  Software  Product  Line  Practice  is  a  service  mark  of  Carnegie  Mellon  University. 

1  The  results  of  some  of  these  workshops  are  documented  in  SEI  reports  [Bass  1997,  Bass  1998,  Bass  1999, 
Bass  2000,  Clements  2001], 
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Subsequent  workshops  were  held  in  successive  years  [Bergey  1999,  Bergey  2000a,  Bergey  2001, 
Bergey  2003,  Bergey  2004,  Bergey  2005], 

At  all  seven  DoD  workshops,  the  SEI  was  encouraged  to  continue  holding  DoD  workshops  and  to 
continue  sharing  best  commercial  and  DoD  practices  through  these  forums.  In  2006  the  workshop 
was  held  as  a  “birds  of  a  feather”  session  in  conjunction  with  the  International  Software  Product 
Line  Conference,  SPLC  2006,  in  Baltimore,  Md.  In  2007,  sponsorship  switched  to  the  Army  Stra¬ 
tegic  Software  Improvement  Program  (ASSIP)  under  the  auspices  of  the  Assistant  Secretary  of 
the  Army  for  Acquisition,  Logistics  and  Technology  [ASA  (ALT)]. This  2009  workshop  was  the 
second  Army  Workshop  sponsored  by  ASSIP. 

1 .2  About  This  Workshop 

The  goals  of  the  February  2009  Army  Software  Product  Line  Workshop  were  to 

•  share  Army  and  DoD  product  line  practices,  experiences,  and  issues,  from  both  development 
and  acquisition  viewpoints 

•  examine  barriers  and  enablers  to  much  broader  adoption  of  software  product  line  practices 
within  the  Army 

•  determine  the  steps  needed  to  make  software  product  line  practices  more  beneficial  and  rele¬ 
vant  to  Army  programs 

•  discuss  ways  in  which  the  ASSIP  can  be  of  assistance 

All  participants  in  this  workshop  were  from  the  DoD  acquisition  and  contractor  community;  there 
was  one  university  affiliate.  They  were  invited  based  on  our  knowledge  of  their  experience  with 
and  commitment  to  software  product  lines  as  either  DoD  system  acquirers  or  DoD  system  con¬ 
tractors.  Together,  the  group  discussed  the  issues  that  form  the  backbone  of  this  report. 

The  format  of  this  workshop  followed  that  of  the  previous  successful  workshops.  Invited  presen¬ 
tations  were  followed  by  a  facilitated  discussion  of  ideas  stimulated  by  the  presentations.  The 
group  agreed  that  this  format  worked  well. 

The  workshop  participants  included 

•  Ceci  Albert,  Acquisition  Support  Program,  SEI 

•  John  Allen,  Nova  Technologies 

•  Ron  Asan,  U.S.  Army,  Sequoyah 

•  John  Bergey,  Research,  Technology,  and  System  Solutions  Program,  SEI 

•  Allan  Bond,  General  Dynamics  C4  Systems 

•  Thomas  Burke,  Bearing  Point 

•  Sholom  Cohen,  Research,  Technology,  and  System  Solutions  Program,  SEI 

•  Patrick  Donohoe,  Research,  Technology,  and  System  Solutions  Program,  SEI 

•  Edward  Dunn,  U.S.  Navy  Naval  Undersea  Warfare  Center  (NUWC) 

•  Terry  Gatewood,  U.S.  Army,  AMRDEC,  C-RAM  Program  Office 

•  Paul  Jensen,  Overwatch  Textron  Systems 
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•  Lawrence  Jones,  Research,  Technology,  and  System  Solutions  Program,  SEI 

•  Brian  Kemper,  U.S.  Army  PEO  STRI 

•  Jerry  Kunert,  U.S.  Army  RDEC  CERDEC  SEC,  Prophet 

•  Reed  Little,  Research,  Technology,  and  System  Solutions  Program,  SEI 

•  Christal  Martir,  Lockheed  Martin  STS  (CTIA) 

•  Michael  McMahon,  CACI  (Sequoyah) 

•  Dana  Miles,  U.S.  Army,  RDECOM  CERDEC  SED 
.  Khuc  Nguyen,  U.S.  Army  PEO  STRI 

•  Linda  Northrop,  Research,  Technology,  and  System  Solutions  Program,  SEI 

•  Roger  Olsen,  Nova  Technologies 

•  Daniel  Oshea,  Lockheed  Martin  STS  (CTIA) 

•  Barbara  Pemberton,  U.S.  Army  PEO  STRI 

•  Rakesh  Rana,  U.S.  Army  Armament  Software  Engineering  Center 

•  Stephen  Rivera,  Advanced  Systems  Technology,  Inc 

•  Philip  Russel,  Northrop  Grumman  Mission  Systems 

•  William  Samper,  U.S.  Army  PEO  STRI 

•  Dirk  Sarner,  U.S.  Army  IAMD  Project  Office 

•  Robert  Schwenk,  U.S.  Army  ASA(ALT) 

•  Les  Simon,  U.S.  Army  PEO  IEW&S 

•  Donald  Snelgrove,  BAE  Systems 

•  Steve  Snell,  SAIC 

•  Michael  Szydlowski,  U.S.  Army  Weapons  Software  Engineering  Center  (WSEC) 

•  James  Todd,  U.S.  Army  PEO  STRI 

•  Damla  Turgut,  University  of  Central  Florida 

1.3  About  This  Report 

This  document  summarizes  the  presentations  and  discussions  from  the  workshop.  This  report 
is  written  primarily  for  those  in  the  DoD  who  are  already  familiar  with  product  line  concepts, 
especially  those  working  on  or  initiating  product  line  practices  in  their  own  organizations. 
Acquisition  managers  and  technical  software  managers  should  also  benefit  from  this  report. 
Those  who  desire  further  background  information  are  referred  to  the  following  resources: 

•  Software  Product  Line  Essentials  [Northrop  2008] 

•  Basic  Concepts  of  Product  Line  Practice  for  the  DoD  [Bergey  2000b] 

•  A  Framework  for  Software  Product  Line  Practice,  Version  5. 0  [Northrop  2009b] 

•  Software  Product  Lines:  Practices  and  Patterns  [Clements  2002] 

The  next  section  of  this  report  contains  a  digest  of  the  presentations.  A  summary  of  the  facilitated 
discussions  follows.  The  report  concludes  with  a  brief  summary. 
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2  Software  Product  Line  Experiences:  A  Digest  of  Participant 
Presentations 


2.1  Introduction  -  Linda  Northrop,  SEI 

Linda  Northrop,  Director  of  the  Research,  Technology,  and  System  Solutions  Program  at  the  SEI, 
began  by  explaining  the  workshop  goals  and  agenda.  She  then  gave  an  overview  of  software 
product  line  practice.  As  previously  noted,  readers  who  would  like  an  explanation  of  the  basics  of 
software  product  lines  should  see  the  references  in  Section  1 .3  or  Linda’s  workshop  slide  presen¬ 
tation  [Northrop  2009a], 

2.2  A  Proactive  Software  Product  Line  Acquisition  Approach  -  John  Bergey,  SEI 

John  Bergey  of  the  SEI  presented  a  proactive  approach  for  acquiring  a  product  line  in  the  DoD 
environment.  A  product  line  approach  presents  some  unique  challenges  to  DoD  programs.  It  in¬ 
volves  adopting  a  new  set  of  practices,  specifying  an  appropriate  division  of  responsibilities,  and 
contracting  with  suppliers  to  develop,  operate,  and  sustain  a  product  line.  The  product  line  aspects 
are  specified  up  front — precluding  opportunistic  attempts  to  initiate  a  product  line  approach  under 
an  existing  contract — so  that  an  appropriate  set  of  requirements  and  SOW  (statement  of  work) 
tasks  can  be  included  in  the  RFP  (request  for  proposal)  and  the  contract. 

2.2.1  Alternative  Acquisition  Approaches  for  Acquiring  Products  via  a  Product  Line 

There  are  three  basic  approaches: 

1.  A  PM  commissions  a  government  organization  to  develop  a  product  line.  This  strategy 
involves  acquiring  a  government- owned  product  line  (production  capability  and  products) 
using  the  in-house  capabilities  of  a  designated  government  acquisition  organization/  An  ex¬ 
ample  is  the  Army’s  Advanced  Multiplex  Test  System  (AMTS)  [Cohen  2007], 

2.  A  PM  commissions  a  contractor  to  develop  a  government-owned  product  line.  This 
strategy  involves  acquiring  a  government-owned  product  line  (production  capability  and 
products)  from  a  contractor. 

3.  A  PM  commissions  a  contractor  to  develop  products  using  the  contractor’s  proprietary 
product  line.  This  strategy  involves  acquiring  products  directly  from  a  contractor  that  has  an 
existing  product  line.  An  example  is  the  Textron  Overwatch  Intelligence  Center  (OIC)  prod¬ 
uct  line  [Jensen  2009], 

The  difficulty  in  executing  these  different  strategies  varies  significantly  and  requires  different  lev¬ 
els  of  management  sophistication  and  technical  skills  on  the  part  of  the  acquisition  organization. 
Related  considerations  include  the  data  rights  to  product  line  artifacts  and  the  risk  of  a  supplier’s 
going  out  of  business. 


This  may  include  using  the  services  of  local  Systems  Engineering  and  Technical  Assistance  (SETA)  contrac¬ 
tors. 
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2.2.2  Conceptual  View  of  a  Product  Line  Acquisition  Example 

The  two  basic  elements  of  a  software  product  line  acquisition  (Figure  1)  to  be  commissioned  are 

1 .  the  development  of  a  product  line  production  capability 

2.  the  subsequent  development  of  a  family  of  software  products  using  that  production 
capability 

The  product  line  production  capability  includes  product  line  core  assets  and  the  production  plan 
that  enables  products  to  be  built  in  a  prescribed  way.  A  production  plan  prescribes  how  the  prod¬ 
ucts  are  produced  from  the  core  assets.  It  includes  the  process  to  be  used  for  building  products  and 
lays  out  the  project  details  to  enable  execution  and  management  of  the  process  (e.g.,  by  including 
such  details  as  the  schedule,  bill  of  materials,  and  metrics).  An  additional  document,  the  product 
line  concept  of  operations  (CONOPS),  describes  the  organizational  approach  for  operating  the 
product  line  effort,  the  organizational  structure,  and  the  roles  and  responsibilities  of  the  stake¬ 
holders  in  product  line  operations.  It  also  describes  the  interconnections  among  those  involved  in 
the  product  line  effort,  including  communication  mechanisms,  decision  and  conflict  resolution 
processes,  a  document  map,  and  any  supporting  Web  site  or  wiki  for  the  product  line. 


XYZ  Concept  of  Operations 


XYZ 

Product  Line 
Production  Capability 


i 


Development 

of 

XYZ  Family  of 
Software  Products 


Performed  by 
Contractor 


XYZ 


Product  Production  P/an 


C  N 

Family  of 
XYZ 

Software 
Products 
V _  _ 


r 


Contract 

Deliverables 


Contractual 


Task 


Figure  1:  Conceptual  View  of  a  Product  Line  Acquisition 

An  example  of  a  contractual  deliverable  that  would  accompany  the  production  capability  is  a 
product  line  practice  description  document.  Illustrative  examples  of  organizational  management, 
technical  management,  and  software  engineering  practices  that  are  applicable  to  a  product  line  can 
be  found  in  the  SEI  framework  [Northrop  2009b]. 

Product  developers  use  the  production  capability  and  its  core  assets  to  develop  specific  products 
in  the  product  line.  A  product  production  plan  documents  how  an  individual  product  will  be  built 
using  the  core  assets  [Chastek  2002b]. 
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2.2.3  Statement  of  Work  (SOW)  Tasks  Example 


Sample  language  was  presented  for  two  SOW  tasks  specifying  the  acquisition  of  (1)  a  software 
product  line  production  capability  and  (2)  a  family  of  software  products  that  are  to  be  developed 
using  the  production  capability.  These  examples  are  starting  points  (i.e.,  stubs)  that  would  need  to 
be  tailored  to  the  product  line  acquisition  strategy  and  the  policies  of  the  acquisition  organization.3 
The  language  example  (italicized)  for  the  SOW  tasks  is  shown  in  the  following  two  sections.  The 
text  fields  delimited  by  the  angle  brackets  would  need  to  be  appropriately  filled  in  by  the  acquisi¬ 
tion  organization. 

2.2.3. 1  Task  1 :  Software  Product  Line  Production  Capability  (PLPC) 

The  contractor  shall  develop  and  sustain  a  comprehensive  software  product  line  production  ca¬ 
pability  throughout  the  life  of  the  contract.  The  specific  requirements  governing  this  capability 
are  described  in  <PLPC-specification>. 

The  contractor  shall  develop  and  deliver  a  comprehensive  product  line  concept  of  operations 
(CONOPS)  document  [<CON OP S-CRDL>]  and  a  product  line  practice  description  document 
[<PLPD-CDRL> ]  that  describe  how  the  product  line  will  operate  from  an  organizational  and 
technical  management  perspective  and  how  it  willfully  accommodate  all  aspects  of  the  develop¬ 
ment  and  sustainment  of  both  the  <XYZ>  core  assets  and  software  products. 

2. 2. 3. 2  Task  2:  Products  in  the  XYZ  Software  Product  Line 

The  contractor  shall  use  the  production  capability  exclusively  to  develop  and  sustain  a  family  of 
<XYZ>  software  products.  A  “software  product ”  is  a  member  of  the  <XYZ>  software  product 
line  that  corresponds  to  a  to-be-deployed  configuration  of  the  <XYZ>.  Each  software  product  is 
to  be  built  using  the  <XYZ>  core  assets  in  accordance  with  a  prescribed  production  plan  and  the 
specified  product  delivery  schedule  [<CDRL-specifying-XYZ-product-deliverables>]. 

The  specific  requirements  governing  the  development  and  sustainment  of  the  software  products  in 
the  <XYZ>  family  of  products  are  described  in  <sp  ecification-f or -XYZ -family -of-sof tw  are - 
products>.  The  <XYZ>  software  products  are  to  be  built  using  the  production  capability  in  ac¬ 
cordance  with  the  CONOPS  and  supporting  practices  described  in  the  product  line  practice  de¬ 
scription  document.  Moreover,  the  products  are  required  to  be  compliant  with  the  product  line 
software  architecture,  which  is  itself  a  core  asset.  The  core  assets  are  to  include  pre-planned  var¬ 
iation  mechanisms  that  allow  each  asset  to  be  customized  to  meet  <XYZ>  product-specific  re¬ 
quirements. 

2.2.4  Overview  of  a  Proactive  Software  Product  Line  Acquisition  Approach 

Developing  a  suitable  acquisition  strategy  is  a  key  consideration  in  adopting  a  product  line  ap¬ 
proach  in  the  DoD.  Of  the  three  approaches  presented  in  Section  2.2.1,  the  most  challenging  is 
approach  #2  (i.e.,  a  PM  commissions  a  contractor  to  develop  a  government- owned  product  line). 
An  example  of  how  to  implement  this  approach  using  SEI  methods  is  depicted  in  Figure  2. 


The  SEI  and  its  representatives  do  not  warrant  RFP/contract  language  (or  other  acquisition  artifacts  and  meth¬ 
ods)  for  use  in  DoD  or  government  acquisitions.  Acquisition  artifacts  are  provided  as  examples  only.  It  is  the  re¬ 
sponsibility  of  the  Contracting  Officer  having  cognizance  over  the  acquisition  to  determine  their  appropriateness 
and/or  suitability  to  a  specific  acquisition  program. 
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Figure  2:  Overview  of  a  Proactive  Product  Line  Acquisition  Approach 

Tliis  example  is  commonly  referred  to  as  a  competitive  down-select.  It  involves  competitively 
awarding  multiple  contracts  (two  or  three)  during  the  initial  acquisition  phase  (Phase  1)  and  then 
conducting  another  “down- select”  to  make  a  single  contract  award  (Phase  2)  to  the  contractor  de¬ 
monstrating  the  “best -value”  solution.4 

These  two  acquisition  phases  are  described  in  more  detail  in  the  following  sections. 

2.2.4. 1  Phase  1  -  The  Competitive  Fly-Off 

The  purpose  of  Phase  1  is  to  reduce  acquisition  risk  by  promoting  competition  over  time.  Without 
Phase  1,  the  contract  award  would  be  based  on  which  potential  supplier  submits  the  best  technical 
proposal,  and  there  would  be  no  tangible  demonstration  of  capabilities  and  approach.  During 
Phase  1  the  participating  suppliers  are  required  to  produce  certain  deliverables  and  participate  in  a 
set  of  prescribed  activities  as  specified  in  the  RFP/contract  (see  Figure  3). 


Best  value  involves  awarding  a  contract  on  the  basis  of  evaluating  cost  and  non- cost  factors  to  select  the  sup¬ 
plier  whose  proposal  offers  the  greatest  (i.e.,  best)  value  to  the  government  in  terms  of  performance,  risk  man¬ 
agement,  cost  or  price,  and  other  factors.  Other  factors  can  include  evaluating  prototypes,  supplier  demonstra¬ 
tions,  and  other  contract  deliverables. 
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Figure  3:  Overview  of  Phase  1  -  The  Competitive  Fiy-Qff 

Prior  to  initiating  Phase  1,  it  is  recommended  that  the  acquisition  organization  conduct  a  Quality 
Attribute  Workshop  (QAW)  to  elicit  and  specify  the  key  quality  attributes  needed  for  system  suc¬ 
cess  (e.g.,  performance,  security,  interoperability,  modifiability,  and  other  non- functional  re¬ 
quirements)  [Barbacci  2003].  It  is  important  that  these  qualities  be  specified  up  front  in  the  PIT 
so  they  can  be  made  known  to  all  the  offerors  since  they  will  have  a  major  influence  on  the  design 
of  the  product  line  architecture. 

In  conjunction  with  the  initial  down-select,  an  architectural  competency  report  can  be  required  as 
part  of  each  offeror's  technical  proposal.  This  has  the  effect  of  “raising  the  bar”  on  the  architec¬ 
tural  skills  that  are  expected  of  each  potential  supplier  and  thus  mitigating  risks  early  because  the 
architecture  is  the  key  to  a  successful  product  line. 

As  part  of  Phase  1,  each  competing  supplier  is  required  to  develop  and  deliver  several  documents 
and  participate  in  some  prescribed  activities.  These  deliverables  and  activities  (in  the  chronologi¬ 
cal  order  shown  in  Figure  3)  include 

1.  a  product  line  concept  of  operations  (CONOPS).  This  deliverable  documents  the  roles  and 
responsibilities  of  the  various  organizational  elements  that  are  involved  in  product  line  op¬ 
erations. 

2.  a  project  management  plan  (PMP)  and  other  traditional  plans.  This  deliverable  requires  that  a 
set  of  traditional  documents,  such  as  a  PMP,  integrated  master  schedule  (IMS),  risk  man¬ 
agement  plan  (RMP),  test  and  evaluation  plan  (TEMP)  and  software  development  plan 
(SDP),  be  expanded  to  address  product  line  aspects  such  as  core  asset  development  and  sus¬ 
tainment. 

3.  a  demonstration  of  key  product  line  practices.  This  scheduled  activity  requires  each  supplier 
to  demonstrate  a  designated  set  of  product  line  practices  so  product  line  strengths  and  chal- 
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lenges  can  be  evaluated.  The  practices  correspond  to  a  selected  subset  of  the  29  practice  ar¬ 
eas  described  in  the  SEI  framework  [Northrop  2009b]. 

4.  a  product  line  practice  description.  This  deliverable  describes  the  specific  product  line  prac¬ 
tices  that  a  supplier  proposes  to  use  as  part  of  its  technical  solution  approach. 

5.  a  software  architecture  description  document.  This  deliverable  requires  each  supplier  to  pro¬ 
vide  a  summaiy  description  of  its  product  line  architecture  and  to  document  the  relevant 
views,  and  add  information  that  applies  to  more  than  one  view.  A  minimum  of  three  types  of 
views  of  the  architecture  are  required:  module  views,  component- and- connector  views,  and 
allocation  views  [Clements  2002b].  This  documentation  is  a  prerequisite  for  conducting  an 
architectural  readiness  review  and  an  evaluation  of  the  product  line  software  architecture. 

6.  an  architectural  readiness  review  (ARR).  This  activity  involves  (1)  walking  through  several 
quality  attribute  scenarios  for  the  purpose  of  validating  the  suitability  of  the  architectural 
views  that  are  part  of  the  architecture  description  document,  and  (2)  determining  the  ade¬ 
quacy  of  the  architecture  documentation  to  support  a  follow-on  software  architecture  evalua¬ 
tion  using  the  SEI  Architecture  Tradeoff  Analysis  Method®  (ATAM®)  [Bass  2003]. 

These  deliverables  and  activities  play  a  key  role  in  the  final  down- select  (i.e.,  source  selection) 
that  results  in  competitively  awarding  a  follow-on  contract  (Phase  2)  to  develop  the  required  pro¬ 
duction  capability  and  associated  products. 

2.2.4. 2  Phase  2  -  The  Software  Product  Line  Development  Effort 

After  the  Phase  2  contract  is  awarded,  there  are  a  number  of  activities  and  deliverables  that  can  be 
specified  up  front  in  the  REP/contract  to  further  reduce  acquisition  risk. 
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Figure  4:  Overview  of  Phase  2-  The  Software  Product  Line  Development  Effort 

These  Phase  2  activities  and  deliverables  (in  the  chronological  order  shown  in  Figure  4)  include 

1.  conducting  an  SEI  Product  Line  Technical  Probe  ^  (PLTP  SM).  This  activity  involves  exam¬ 
ining  an  organization's  readiness  to  adopt  or  ability  to  succeed  with  a  software  product  line 
approach.  The  probe  is  a  diagnostic  tool  that  utilizes  the  SEI  framework  [Northrop  2009b]  as 
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a  reference  model.  The  results  of  the  probe  are  a  set  of  findings  that  portray  an  organization's 
strengths  and  challenges  with  regard  to  a  product  line  approach. 

2.  conducting  an  architecture  evaluation.  This  activity  involves  evaluating  the  software  product 
line  architecture  relative  to  quality  attribute  goals  using  the  SEI  ATAM.  The  evaluation  re¬ 
sults  in  the  early  discovery  and  identification  of  risks  (and  risk  themes)  in  the  architectural 
design  so  the  development  contractor  can  develop  a  plan  for  mitigating  them  in  a  timely  fa¬ 
shion,  thus  avoiding  extensive  and  costly  rework  downstream.  If  at  all  possible,  the  architec¬ 
ture  evaluation  should  be  completed  prior  to  the  preliminary  design  review  (PDR).  This  al¬ 
lows  the  PDR  team  to  focus  on  the  discovered  risks  and  how  the  development  contractor 
plans  to  mitigate  them,  as  opposed  to  performing  the  traditional  perfunctory  reviews  that  are 
characteristic  of  a  PDR.  Bergey  describes  the  benefits  of  being  proactive  and  conducting  a 
software  architecture  evaluation  early  in  the  contract  performance  phase  of  a  DoD  acquisi¬ 
tion  along  with  comprehensive  guidance  on  how  to  do  it.5 

3.  scheduling  a  core  asset  demo.  This  activity  requires  the  development  contractor  to  demon¬ 
strate  the  proper  functioning  and  operation  of  a  selected  set  of  assets  that  are  critical  to  prod¬ 
uct  line  success  and  to  demonstrate  a  production  capability  by  using  core  assets  to  develop  a 
test  product. 

4.  concurrent  delivery  of  two  production  plans.  This  deliverable  calls  for  the  concurrent  deliv¬ 
ery  of  two  production  plans  as  a  precursor  to  the  next  milestone  event  that  calls  for  the  con¬ 
current  delivery  of  two  corresponding  products  that  are  to  be  built  in  accordance  with  their 
respective  production  plans. 

5.  concurrent  delivery  of  two  products  in  the  product  family.  This  deliverable  calls  for  the  con¬ 
current  delivery  of  two  products  that  are  required  to  be  built  from  a  common  set  of  core  as¬ 
sets  at  a  fixed  price.  This  requirement  precludes  the  developer  from  adopting  a  “clone-and- 
own”  approach  that  would  result  in  stovepiped  products  that  would  have  to  be  separately 
maintained  as  distinct  products. 

6.  coordinated  delivery  of  a  new  production  plan  preceding  the  corresponding  development  and 
delivery  of  a  new  product  in  the  product  family.  Unlike  the  first  two  products  these  would 
most  likely  be  developed  on  a  cost-plus-award-fee  basis  under  a  delivery  order  IDIQ  (indefi¬ 
nite  delivery /indefinite  quantity)  contract. 

These  activities  and  deliverables  are  illustrative  examples  only.  They  show  how  SEI  architecture 
and  product  line  methods  can  be  effectively  used  to  mitigate  acquisition  risks  in  a  product  line 
effort.  To  determine  the  activities  and  deliverables  that  would  actually  be  most  effective,  the  rec¬ 
ommended  practice  is  to  conduct  an  acquisition  planning  workshop  with  stakeholders  early  in  the 
acquisition  planning  phase  (See  Section  2.3). 

In  summary,  in  a  proactive  acquisition  approach  desired  product  line  activities  and  deliverables 
are  preplanned  and  integrated  up  front  in  the  RFP.  Unless  an  acquisition  organization  takes  a  pro¬ 
active  approach,  it  will  not  have  an  effective  means  for  managing  a  product  line  contract  and  per¬ 
forming  its  technical  oversight  and  contract  monitoring  responsibilities. 


Bergey,  J.  A  Proactive  Means  for  Incorporating  an  ATAM  Software  Architecture  Evaluation  in  a  DoD  System 
Acquisition.  Software  Engineering  Institute,  Carnegie  Mellon  University.  To  be  published. 
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2.3  An  Approach  to  Software  Product  Line  Acquisition  Planning  -  Larry  Jones,  SEI 

Larry  Jones  presented  an  overview  of  an  SEI  approach  to  product  line  acquisition  planning  that 
has  proven  useful  in  a  number  of  engagements.  SEI  experience  has  shown  that  all  too  often  acqui¬ 
sition  planning  is  given  inadequate  attention;  the  downstream  consequences  of  this  are  frequently 
painful.  Because  of  its  greater  complexity,  a  software  product  line  acquisition  calls  for  especially 
careful  planning. 

The  SEI  Acquisition  Planning  Workshop  is  a  one  or  one-and-a-half  day  facilitated  technical  inter¬ 
change  among  key  acquisition  stakeholders.  It  should  occur  early  in  the  acquisition  life  cycle,  in 
order  to  achieve  its  ultimate  purpose:  to  reduce  software  acquisition  risk 

I¥ocedurally,  an  SEI  team  works  in  advance  with  the  acquisition  organization’s  point  of  contact  to 
ensure  that  the  proper  stakeholders  are  committed  to  attend,  that  all  presentations  are  prepared  in 
advance,  and  that  logistical  arrangements  are  in  order.  Additionally,  it  is  very  useful  if  a  set  of  key 
acquisition  challenges  can  be  drafted  prior  to  the  workshop  by  the  SEI  team  and  the  point  of  con¬ 
tact.  A  well-thought-out  set  of  acquisition  challenges  allows  the  SEI  team  to  craft  a  set  of  ques¬ 
tions  to  guide  the  discussions.  During  the  workshop,  one  of  the  SEI  team  serves  as  a  facilitator 
and  another  as  a  recorder  to  capture  key  discussion  points,  risks,  and  issues. 


The  conceptual  phases  of  the  woikshop  are  illustrated  in  Figure  5  and  a  sample  agenda  is  given  in 
Table  1. 


Understand 


vM  capture  risks, 
key  issues  and 
points  of  consensus 
throughout 


Figure  5:  Conceptual  Phases  of  a  Product  Line  Acquisition  Planning  'Workshop 
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Table  1:  Sample  Planning  Workshop  Agenda 


Time 

Topic 

Responsibility 

0800-0830 

1.  Welcome  &  Introductions 

SEI/PO 

0630-0900 

2.  System  overview  end  product  line  vision 

PO 

0900-0915 

3.  Overview  of  acquisition  organization  and  stakeholders 

PO 

0915-0930 

4.  Acquisition  Hfecycle  -  program  status 

PO 

0930-1000 

5.  Elicitation  of  acquisition  drivers,  constraints  and  issues 

SEI  Facilitation 

1000-1015 

Break 

1015-1045 

6.  Basic  product  line  acquisition  approaches 

SEI 

1045-1100 

7.  Overview  of  system  acquisition  challenges 

SEI 

1100-1200 

8.  Elaboration*  of  Challenge  #1 

SEI  Facilitation 

1200-1300 

Lunch 

PO 

1300-1400 

9.  Elaboration*  of  Challenge  #2 

SEI  Facilitation 

1400-1500 

10.  Elaboration*  of  Challenge  #3 

SEI  facilitation 

1500-1515 

Break 

1515-1615 

11.  Elaboration*  of  Challenge  #4 

SEI  Facilitation 

1615-1700 

12.  Review  and  Next  Steps 

SEI  Facilitation 

*  hckidB  nriifB  of  risks  IwlsaiB  nHwi  s —w  and  of 


The  purpose  of  the  Understand  phase  (steps  1-5  in  the  agenda)  is  to  bring  all  participants  up  to 
speed  on  the  system  to  be  acquired  and  the  status  of  the  acquisition.  The  acquisition  office  will 
first  make  presentations  that 

•  overview  the  system,  its  mission  need,  and  context 

•  describe  a  high-level  vision  for  the  product  line 

•  show  the  structure  of  the  acquisition  organization  and  its  relationships  to  other  key 
stakeholders 

•  summarize  the  status  of  the  acquisition  and  the  schedule 

The  SEI  team  then  makes  a  presentation  describing  the  different  basic  product  line  acquisition 
approaches  that  might  be  taken  (Step  5  in  the  agenda).  These  approaches  are  given  in  Section 

2.2.1. 

The  Elicit  phase  (Step  6  in  the  agenda)  is  a  facilitated  session  in  which  acquisition  drivers  and 
constraints  are  elicited.  These  are  recorded  along  with  any  risks,  issues,  or  acquisition  challenges 
that  arise  during  the  discussion.  Typical  sources  of  drivers  and  constraints  include 

•  externally  imposed  constraints  (e.g.,  joint  service  considerations,  interoperability  require¬ 
ments,  imposed  deadlines) 

•  constraints  adopted  by  the  acquisition  organization  (e.g.,  dates  chosen  for  RFP  release) 

•  constraints  imposed  by  other  stakeholders  (e.g.,  flight  certification  requirements, 
backward-compatibility  requirements) 


Once  the  background,  constraints,  and  drivers  have  been  shared,  the  group  is  ready  for  the  Ex¬ 
plore  phase  (Steps  7-11  in  the  agenda).  First,  the  key  acquisition  challenges  determined  prior  to 
the  workshop  are  overviewed.  Then  each  is  explored  in  turn  through  a  facilitated  discussion.  The 
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recorder  captures  discussion  points,  risks  and  issues.  These  key  challenges  will  depend  on  the 
context  of  a  particular  acquisition.  They  are  derived  beforehand  through  interaction  between  the 
SEI  team  and  the  acquisition  organization  point  of  contact. 

As  an  example,  consider  the  situation  where  an  existing  product  line  effort  will  serve  as  the  basis 
for  a  new  competitive  acquisition  for  a  follow-on  family  of  systems.  A  set  of  key  challenges  for 
this  example  might  be 

•  How  to  support  ongoing  product  development  and  sustainment 

•  How  to  take  possession  of  and  transition  existing  assets  for  future  use 

•  How  to  initiate  the  new  competitive  acquisition 

•  How  to  manage  the  overall  program  office  commitments 


For  each  of  these  challenges  a  set  of  questions  to  drive  the  discussion  are  developed  prior  to  the 
workshop.  Sample  discussion  questions  for  the  challenge,  “How  to  support  ongoing  product  de¬ 
velopment  and  sustainment”  could  include 

•  What  is  the  scheduled  life-time  for  currently  deployed  systems?  What  support  will  be  neces¬ 
sary  during  this  period?  Is  there  a  phase-out  planned? 

•  Are  there  dependencies  between  this  sustainment  and  the  new  acquisition? 

•  How  long  is  the  current  contract  in  effect?  Is  an  extension  necessary  to  provide  continuity  of 
support?  Should  the  scope  of  work  be  reduced  (e.g.,  just  focus  on  critical  fixes)? 


Examples  of  discussion  questions  for  the  challenge  “How  to  take  possession  of  and  transition  ex¬ 
isting  assets  for  future  use”  could  include 

•  Have  we  validated  the  government  data  rights? 

•  Do  we  have  an  inventory  of  transitionable  assets?  Is  additional  effort  necessary  to  package  the 
assets? 

•  Do  we  have  the  necessary  training  materials? 


To  craft  a  set  of  discussion  questions  it  is  useful  to  consider  the  following  general  questions. 

•  What  needs  to  be  done? 

•  When  will  it  have  to  be  done? 

•  Who  will  be  responsible  for  doing  it? 

•  Where  are  the  unknowns? 

•  What  will  be  difficult  about  different  approaches? 

•  What  is  realistic  in  terms  of  effort  and  time? 

•  What  constrains  our  work? 

•  How  do  we  make  sure  the  work  is  done  satisfactorily? 

•  How  can  we  avoid  past  problems? 
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Throughout  the  discussion,  stated  risks  and  issues  are  recorded.  Examples  of  such  a  transcription 

could  include 

•  There  is  no  common  understanding  of  the  scope  of  the  effort. 

•  Because  of  the  way  the  approach  is  currently  planned,  the  government  will  be  thrust  into  the 
role  of  system  integrator. 

•  The  schedule  is  unrealistic  in  <these  areas>. 

•  We  need  to  determine  if  a  competitive  down-select  is  the  preferred  approach. 

•  The  envisioned  acquisition  approach  is  likely  to  result  in  a  family  of  “clone- and- own”  prod¬ 
ucts  rather  than  a  product  line. 

•  The  Request  for  Information  needs  to  be  more  fully  developed. 

•  The  program  office  does  not  have  sufficient  staff  to  accomplish  the  tasks  we  have  identified. 

•  There  is  a  gap  between  what  the  current  contractor  is  obliged  to  deliver  and  what  is  needed 
for  subsequent  life- cycle  support. 

•  Non-functional  (i.e.,  quality)  requirements  are  inadequately  understood  and  inadequately  do¬ 
cumented. 


Finally,  the  results  are  reviewed  and  follow-on  actions  are  assigned  in  the  Focus  phase  (Step  1 1  in 
the  agenda). 

•  The  SEI  has  received  positive  feedback  from  the  workshops  conducted  thus  far.  Reported 
benefits  include 

improved  identification  of  stakeholders 
improved  communication  among  acquisition  stakeholders 
improved  understanding  of  acquisition  risks 
a  better  basis  for  a  successful  acquisition 

2.4  The  Overwatch  Intelligence  Center  Software  Product  Line  -  Paul  Jensen, 
Overwatch  Textron  Systems 

Paul  Jensen,  chief  architect  at  Overwatch  Textron  Systems,  provided  an  update  of  the  Overwatch 
Intelligence  Center  (OIC)  software  product  line  [Jensen  2009],  The  product  line  provides  intelli¬ 
gence  collection,  processing,  and  visualization  capabilities  to  U.S.  government  departments  and 
agencies.  The  adoption  of  a  product  line  approach  to  software  development  was  spurred  by  the 
recognition  of  the  inadequacy  of  the  single-system  opportunistic  reuse  approach,  and  by  the  diffi¬ 
culty  of  achieving  synergy  across  the  suite  of  products  that  arose  from  the  company’s  acquisition 
of  several  software  companies.  Overwatch  developed  a  business  case  to  show  that  a  software 
product  line  (SPL)  was  an  appropriate  response  to  these  challenges.  The  goal  was  to  have  every 
customer  solution  and  every  product  be  a  part  of  the  SPL  within  four  years.  Improved  time  to 
market  was  also  a  driver  for  adopting  the  product  line  approach. 
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The  transition  to  the  product  line  approach  began  in  2003,  with  support  from  the  CEO,  VP  of  en¬ 
gineering,  and  chief  architect.  A  specific  customer  delivery  was  targeted  to  be  the  first  member  of 
the  product  line.  The  first  product  from  the  OIC  SPL  was  released  in  2005.  Multiple  systems  have 
since  been  fielded  from  the  product  line,  including  two  licensed  products:  an  all-source  analysis 
system  and  a  signal  intelligence  (SIGINT)  system.  The  first  program-of-record  system  was  fielded 
in  2008. 

One  early  challenge  for  Overwatch  was  coming  up  with  a  funding  model  that  would  support  core 
asset  development  (a  common  problem  for  organizations  making  the  transition  from  delivery  of 
customer-funded  individual  products  to  the  delivery  of  members  of  a  product  line  built  from 
shared  resources).  A  second  challenge  was  that  resources  were  often  pulled  from  the  core  asset 
development  effort  to  plug  critical  gaps  in  customer- specific  product  development  efforts.  The 
company  had  to  change  its  funding  and  organizational  model  to  one  in  which  the  responsibility 
for  developing  core  assets  was  shared  across  the  product  development  and  asset  development 
roles. 

Another  issue  for  Overwatch  was  the  creation  of  a  true  product  line  architecture.  In  the  early  SPL 
architecture  the  organization  neglected  to  address  issues  related  to  the  product  engineering  or  as¬ 
sembly  aspects  of  the  product,  and  did  not  define  much  of  the  infrastructure  that  is  needed  to  op¬ 
erate  a  product  line  efficiently.  Overwatch  also  had  an  extensive  legacy  base  of  some  10  million 
lines  of  code  that  it  wanted  to  mine  for  reuse  in  the  product  line;  the  lack  of  experience  in  domain 
analysis  and  the  lack  of  an  architecture  to  support  a  product  line  hampered  the  creation  of  core 
assets  from  the  legacy  base.  To  solve  these  problems,  the  company  created  a  new  architecture 
called  Viper.  Viper  is  a  licensable  framework  that  enables  the  creation  of  members  of  the  product 
line  by  composing  or  assembling  the  reusable  software  assets.  It  is  the  core  of  the  OIC  SPL,  and  it 
includes  a  software  development  kit  to  support  product  engineering  [Jensen  2007]. 

The  lack  of  experience  with  domain  analysis  (a  typical  situation  for  organizations  moving  from 
opportunistic  to  systematic  reuse)  also  meant  that  Overwatch  struggled  initially  with  identifying 
the  commonality  and  variation  across  current  and  proposed  products  for  its  product  line.  Varia¬ 
tions  in  non-software  core  assets  also  caused  some  problems;  requirements  test  artifacts,  for  ex¬ 
ample,  weren’t  initially  controlled  as  assets.  These  problems  have  all  been  overcome. 

Remaining  challenges  include 

•  the  difficulty  of  quantifying  the  value  proposition  of  product  lines  in  an  Army  acquisition 
context 

•  rapidly  changing  requirements  due  to  the  nature  of  the  current  fight  (which  lessens  the  accu¬ 
racy  of  domain  analysis  and  introduces  a  high  rate  of  change  in  the  core  assets) 

•  the  overall  difficulty  of  balancing  SPL  objectives  and  acquisition  needs  (e.g.,  mandated 
events  affecting  the  schedule,  multi-program  side  effects) 

The  product  line  approach  has  yielded  several  benefits.  The  all-source  analysis  product  was  built 
in  fewer  than  90  days,  with  an  estimated  time-to-market  improvement  of  about  2.5.  More  than  10 
customer  deliveries  have  been  made  with  members  of  the  product  line,  and  reuse  of  core  assets 
across  two  SIGINT  system  deliveries  is  about  70%.  Anecdotal  evidence  suggests  benefits  in  addi¬ 
tional  areas,  such  as  product  quality  and  speed  of  integration,  but  Overwatch  is  still  working  on 
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quantifying  these  values.  Overall,  the  company  made  some  early  missteps,  but  is  now  on  track 
with  the  OIC  SPL  and  has  achieved  a  faster  time  to  market  and  more  efficient  and  cost-effective 
reuse. 

The  lessons  learned  from  the  Overwatch  experience  with  product  line  adoption  include  the  fol¬ 
lowing: 

•  Senior  management  support  is  critical. 

•  Carefully  match  the  organizational  model  to  the  funding  model. 

•  Product  line  architecture  is  essential. 

•  Address  product  line  requirements  up  front. 

•  Put  processes  in  place  to  perform  domain  analysis  activities  as  early  in  the  project  life-cycle 
as  possible. 

•  Agile  development  on  product  line  assets  is  problematic;  it’s  better  suited  to  product  devel¬ 
opment. 

For  the  future,  the  plans  are  to  improve  processes,  tools,  and  metrics  while  maintaining  current 
products.  A  second  program-of-record  system  will  be  fielded  in  2009.  There  will  also  be  a  move 
to  a  next-generation  architecture  incorporating  cloud  computing  (infrastructure  as  a  service,  plat¬ 
form  as  a  service,  and  software  as  a  service)  and  enabling  composite  applications  composed  of 
independent  but  collaborating  modules. 

2.5  The  LVC-IA  Software  Product  Line  -  Brian  Kemper,  U.S.  Army  PEO  STRI 

Brian  Kemper,  chief  engineer,  LT2  PEO  STRI,  presented  an  overview  of  product  line  practices 
and  acquisition  approaches  employed  by  PEO  STRI.  Currently  PEO  STRI  has  three  product-line- 
related  initiatives  involving  each  of  the  three  domains  supported  by  its  organization:  live,  con¬ 
structive,  and  virtual  simulation.  The  Live,  Virtual,  Constructive  Integrating  Architecture  (LVC- 
IA)  provides  the  foundational  structure  and  framework  for  linking  systems  in  these  domains  into 
the  warfighter’s  integrated  training  environment. 

In  the  live  domain,  the  Common  Training  Instrumentation  Architecture  (CTIA)  provides  a  prod¬ 
uct  line  architecture  that  supports  live  instrumentation.  Tactical  Engagement  Simulation  Systems 
(TESS),  targetry,  domain-specific  services,  and  associated  equipment  for  live  training  within  the 
Army’s  doctrine-based  training  process.  It  is  the  core  of  the  Live  Training  Transformation-Family 
of  Training  Systems  (LT2-FTS)  architectural  framework  and  LT2  repository  of  reusable  assets. 
The  repository,  located  in  the  LT2  portal,  is  intended  to  facilitate  delivery  of  integrated  and  inter¬ 
operable  training  solutions  for  live  collective  training  across  the  home  station,  maneuver  Combat 
Training  Centers  (CTCs),  and  deployed  and  joint  training  domains.  The  live  training  domain 
needed  a  common  architecture;  to  meet  this  requirement  the  LT2  Initial  Capability  Document 
(ICD)  provides  a  Joint  Capability  Integration  and  Development  System  (JCIDS)  acquisition  doc¬ 
ument  and  product  line  approach. 

In  the  constructive  domain,  the  One  Semi- Automated  Forces  (OneSAF)  program  is  the  current 
product  line  initiative.  OneSAF  is  a  next-generation,  entity- level  simulation  that  supports  both 
Computer  Generated  Forces  (CGF)  and  Semi- Automated  Forces  (SAF)  applications.  This  enables 
it  to  support  a  wide  range  of  U.S.  Army  brigade-and-below  constructive  simulations  and  virtual 
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simulators.  OneSAF’s  requirements  were  to  develop  a  product  that  needed  to  be  composable  for 
use  by  multiple  customers.  There  was  also  the  potential  to  reuse  the  OneSAF  product  across  other 
domains.  This  contributed  to  a  business  case  in  which  it  made  sense  to  use  a  software  product  line 
approach. 

In  the  virtual  domain,  the  Synthetic  Environment  Core  (SE  Core)  is  the  product  line  architecture 
example.  The  SE  Core  encompasses  the  Army’s  overarching  strategy  of  developing  virtual  simu¬ 
lation  systems  that  support  warfighter  training.  The  SE  Core  ORD  (Operational  Requirements 
Document)  has  interoperability  and  common  component  requirements.  It  was  determined  that  a 
product  line  architecture  approach  was  the  appropriate  strategy  to  satisfy  these  requirements. 

Despite  the  fact  that  each  of  these  initiatives  has  had  a  different  acquisition  strategy,  each  is 
guided  by  the  application  of  product  line  practice,  strategic  reuse,  and  common  standards  initia¬ 
tives. 

Brian  discussed  several  benefits  of  a  software  product  line  approach: 

•  SO  A  (Service  Oriented  Architecture)  is  starting  to  become  important  in  his  organization’s 
work,  and  many  issues  about  adopting  SOA  have  already  been  dealt  with  in  the  product  line 
work  (e.g.,  reuse,  acquisition  culture,  and  budget  issues). 

•  The  product  line  approach  is  helping  with  interoperability  of  systems  that  are  all  built  using 
the  same  product  line. 

•  In  the  live  domain,  they  have  a  mature  product  line  with  a  lot  of  core  assets,  so  it  is  easier  to 
convince  programs  and  program  managers  to  use  it.  In  the  “gaming -virtual”  domain  the  prod¬ 
uct  line  approach  isn’t  as  mature,  so  the  product  line  is  a  harder  sell. 

•  The  virtual  domain/SE  Core  is  currently  benefiting  from  the  OneSAF  software  product  line 
development  approach  in  the  utilization  of  OneSAF  to  replace  legacy  virtual  simulation  sys¬ 
tems.  This  improves  interoperability  across  the  domain  and  supports  the  overarching  LVC  ob¬ 
jectives. 


Brian  offered  several  software  product  line  challenges  and  lessons  learned: 

•  The  current  acquisition  culture  does  not  facilitate  product  line  approaches.  IPTs  (Integrated 
Product  Teams),  trust,  good  software  development  processes,  core  structure,  buy-in  by  high- 
level  management,  and  enforcement  are  necessary  to  maintain  and  mature  the  product  line. 

•  Currently,  on  the  contractor  side,  product  line  related  success  is  due  to  sophisticated  technical 
and  management  talent;  the  product  line  software  architect  is  usually  a  dedicated  person  who 
reports  to  the  program  manager. 

•  On  the  government  side,  there  must  be  good  architecture  expertise;  the  program  office  must 
have  the  software  architecture  capability  in  house,  otherwise  the  program  office  can’t  appro¬ 
priately  manage  industry  software  architecture  development  teams  and  management  initia¬ 
tives. 

•  There  is  a  need  to  adopt  acquisition  policies  and  processes  improvements  that  enable  product 
line  funding  and  execution  based  on  domain  requirements  commonality. 


17  |  CMU/SEI-2009-TR-01 2 


•  The  various  product  line  initiatives  at  PEO  STRI  recognize  the  importance  of  strong  configu¬ 
ration  management  (CM)  processes.  As  multiple  programs  begin  to  leverage  and  extend  the 
core  assets,  changes  must  be  coordinated  across  the  programs. 

•  Product  line  scope  always  seems  to  be  a  “moving  target”  to  accommodate  budget  cuts  and 
efficiencies.  So  there  is  a  need  to  baseline  product  line  requirements  and  to  be  ready  to  evolve 
the  domain  over  time. 

•  There  needs  to  be  effective  management  of  the  product  line.  Brian  pointed  out  that  good  met¬ 
rics  are  important  to  determine  cost  and  ROI  (return  on  investment).  LT2  uses  a  briefing  for 
management  showing  payoff  with  use  of  the  product  line  approach.  There  are  programs  that 
are  now  being  developed  with  an  80-90%  reuse  of  core  assets.  These  successes  make  it  easier 
to  begin  to  show  hard  cost/savings  numbers  (both  in  development  time  and  cost).  It  is  also 
necessary  to  show  how  the  product  line  approach  will  not  adversely  affect  the  program  sche¬ 
dules.  The  adoption  of  this  strategy  requires  complete  buy-in  from  product  managers. 

•  There  is  a  need  to  achieve  industry  buy-in  and  participation  with  the  specific  government 
product  line  concept.  This  involves  clearly  defining  how  industry  can  profit  from  the  ap¬ 
proach  (because  companies  could  perceive  that  this  affects  their  bottom  line  adversely). 

•  The  PEO  has  a  strategy  of  moving  away  from  system  development  containing  proprietary 
solutions  and  data.  Commercial  off-the-shelf  (COTS)  solutions  have  a  place  in  the  product 
line  approach  provided  they  can  be  integrated  into  the  product  line  architecture. 

•  The  PEO  is  developing  its  standard  interfaces  (for  interoperability)  in  cooperation  with  indus¬ 
try.  The  PEO  encourages  COTS  vendors  to  invest  in  order  to  become  compliant  with  the 
standards. 

•  Education  is  essential  for  all  stakeholders.  Brian  said,  “AS SIP  product  line  education  options 
provide  a  valuable  opportunity  to  help  get  stakeholders  in  line  with  the  overarching  architec¬ 
ture  objectives.” 

•  Early  successes  are  important.  It  is  easy  for  the  product  line  domain  analysis  team  to  “go  a 
mile  wide  and  an  inch  deep”  and  begin  to  lose  momentum.  It  is  important  to  effectively  and 
quickly  establish  an  initial  domain  analysis;  this  has  enabled  PEO  STRI  to  get  some  quality 
software  core  assets  out  to  the  product  builders  in  a  reasonable  time  frame. 


Brian  emphasized  that  up-front  investment  is  necessary  for  a  successful  product  line.  The  PM 
shops  must  commit  to  contributing  to  the  base  product  line  work  (core  asset  set  and  product  line 
architecture).  In  the  live  domain  their  acceptance  of  the  need  for  that  up-front  investment  is  now 
starting  to  reap  the  benefits. 

The  way  the  Army  usually  works  is  that  money  comes  into  programs,  and  these  programs  usually 
don’t  have  enough  dollars  to  do  what  they  need  to  do  functionally.  So  it  is  difficult  to  convince 
the  programs  to  provide  the  up-front  investment  required  to  support  a  product  line  approach,  es¬ 
pecially  when  the  PM  may  not  realize  the  benefit  until  a  year  or  two  down  the  road.  The  challenge 
is  to  find  a  way  to  convince  PMs  to  embrace  the  product  line  approach  and  apply  funds  to  the 
product  line  support.  Organizational  buy-in  at  the  PEO  and  0-6  levels  is  essential  to  the  successes 
seen  across  live,  virtual,  and  constructive  domains  at  PEO  STRI. 
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Brian  provided  several  examples  of  product  line  acquisition  enablers  for  the  LT2  program: 

•  a  specific  LT2  product  line  funding  line 

•  an  Integration  and  Interoperability  Advisory  Board  (12 AB)  that 

facilitates  the  integration  and  interoperability  of  PEO  STRI  training  systems  combining 
technical  and  programmatic  perspectives 

provides  architecture  and  technical  recommendations  across  the  PEO  by  applying  a  ho¬ 
rizontal/vertical  integration  approach 

leverages  PEO  product  line  assets  across  the  PEO  to  support  interoperability  across  the 
Army  and  other  services 

•  life-cycle  configuration  management  as  a  key  to  maintaining  product  line  integrity 

•  a  project  manager  with  “product  line  vision”  who  is  willing  to  think  outside  the  “acquisition 
box”  while  making  sure  always  to  address  how  this  benefits  the  soldier 

Brian  concluded  with  a  quote  from  Dr.  James  T.  Blake,  Program  Executive  Officer,  U.S.  Army 
PEO  STRI,  that  demonstrates  the  organizational  commitment  to  software  product  lines. 

The  current  operational  environment  requires  quickly  adaptable  systems  for  the  Warfighter. 
To  better  support  this  challenging  environment,  PEO  STRI  is  adopting  the  deliberate  and 
disciplined  tenets  of  Product  Line  Acquisition  to  maximize  adaptability. 

Product  Line  Acquisition  tenets  emphasize  re-use  of  components,  synchronizing  the  produc¬ 
tion  of  products,  and  seamless  interoperability  between  components,  products,  and  systems. 
These  tenets  bring  the  Live,  Virtual,  and  Constructive  domains  together  to  facilitate  interop¬ 
erability  and  integration  (12). 

2.6  Electronic  Warfare  Product  Line  -  Don  Snelgrove,  BAE  Systems 

Don  Snelgrove,  Director  of  the  Product  Centered  Organization  of  BAE  Systems,  Electronic  Solu¬ 
tions,  Nashua,  N.H.,  made  a  presentation  on  behalf  of  the  Electronic  Solutions  line  of  business. 
This  product  line  is  under  development  at  BAE  Systems,  Hudson,  N.H. 

BAE  Systems  develops  a  wide  range  of  EW  (electronic  warfare)  products  for  advanced  defense 
and  aerospace  systems.  Recent  examples  include  the  F-22  and  Joint  Strike  Fighter.  EW  products 
may  be  used  to  detect,  identify,  locate,  signal  process,  perform  threat  analysis,  or  jam  other  radio 
frequency  (RF)  emissions.  Customers  include  many  DoD  services,  joint  services,  and  foreign  mil¬ 
itary. 

Most  of  these  applications  implement  SIGINT  (signal  intelligence)  functionality.  On  the  front 
end,  these  systems  sense  RF  emissions,  capture  those  signals,  process  them,  and  provide  the  in¬ 
formation  for  display  in  a  cockpit  or  ground  station.  BAE  is  responsible  for  the  entire  product — 
software,  firmware,  and  hardware. 

2.6.1  Issues  and  Challenges  of  the  Product  Line  Approach 

BAE’s  product  line  effort  emerged  in  the  late  1990s  from  a  series  of  cost-plus  contracts  that  ended 
when  higher-than-projected  costs  were  reached.  Recognizing  commonality  across  many  of  their 
systems,  BAE  Systems  created  a  software  architecture  for  SIGINT.  The  initial  architecture  did  not 
cover  the  entire  EW  mission  area — funding  was  not  sufficient  to  make  this  feasible.  However, 
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BAE  Systems  adopted  a  component  strategy  looking  at  the  geo- location,  signature  identification, 
and  other  high-value  domains.  Development  of  the  common  architecture  used  in-house  project 
funds,  with  support  from  the  vice  president  within  BAE  Systems  who  also  directed  use  of  the 
common  architecture. 

Over  a  series  of  developments,  BAE  Systems  added  building  blocks  and  reached  a  critical  mass  in 
terms  of  capabilities  for  the  EW  mission  area.  Building  on  this,  BAE  Systems  established  a  prod¬ 
uct-centered  organization  that  considers  not  only  software,  but  also  firmware  for  field  program¬ 
mable  gate  array  (FPGA)  hardware.  This  firmware  tends  to  be  higher  cost  and  more  difficult  to 
port  than  software.  So  extending  portability  attributes  through  the  common  architecture  is  a  sig¬ 
nificant  gain  in  terms  of  non-recurring  expenses. 

BAE  Systems’  product  line  approach  involves  coupling  project  support  with  focused  IRAD  (in¬ 
ternal  research  and  development)  to  enhance  the  core  asset  base.  Decisions  about  where  to  apply 
the  IRAD  effort  come  from  marketing,  which  identifies  the  emerging  trends  and  seeks  to  address 
them  in  advance  of  actual  projects.  Under  this  approach,  the  core  asset  base  includes  software  + 
hardware  (not  antennas)  +  firmware.  A  typical  program  is  now  a  mix  of  core  assets  plus  new 
software,  hardware,  or  firmware. 

2.6.2  Impact  of  the  Product  Line 

The  collection  systems  within  the  product  line  now  have  a  broad  customer  base.  BAE  Systems 
sees  its  objectives  as  refreshing  and  extending  the  core  asset  base  for  hardware  and  software.  A 
typical  project  anticipates  very  high  reuse  of  core  assets,  although  actual  reuse  percentages  can 
vary  depending  upon  the  project  type.  One  difficulty  is  in  measuring  the  profit  achieved  to  in¬ 
vestment  made — how  critical  is  the  core  asset  base  to  the  organization? 

The  product  line  approach  reduces  recurring  costs.  It  also  improves  system  producibility.  Com¬ 
mon  software  contributes  to  common  hardware  buys,  increasing  the  numbers  of  buys  of  circuit 
cards  for  multi-program  use,  rather  than  individual  program  use.  This  provides  leverage  across 
hardware  and  firmware.  Programs  also  have  realized  gains  in  integrated  engineering — where 
hardware  design  and  manufacturing,  and  software  development  work  together  to  maximize  reuse. 

BAE  Systems  has  developed  a  concept  referred  to  as  “X  +  Y  +  Z.”  This  concept  categorizes  soft¬ 
ware  on  a  product  as  being  either  X  (from  core  assets),  Y  (not  currently  from  core  assets,  but 
showing  potential),  or  Z  (not  feasible  as  core  assets).  A  typical  bid  closely  scrutinizes  those  capa¬ 
bilities  in  the  Y  for  evolution  to  core  assets.  The  term  delta-Y,  or  AT,  refers  to  those  capabilities 
that  with  add-on  funds,  can  be  made  a  core  asset. 

BAE  Systems  has  not  solved  all  the  problems  of  using  core  assets.  Some  of  the  issues  being  grap¬ 
pled  with  are 

•  personnel  sharing.  Programs  need  common  resources  and  sharing  of  personnel,  so  personnel 
no  longer  “belong”  to  a  project  but  are  a  part  of  the  cadre  of  engineers  in  the  pro  duct- centered 
organization. 

•  security.  Different  project  security  levels  use  shared  code.  A  question  is,  “How  do  you  main¬ 
tain  software  code  at  security  levels  dictated  by  varying  security  classification  guides?” 
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•  technology  refresh.  Hardware  technology  may  last  for  four  years.  A  challenge  is  to  select  the 
new  technology  with  IRAD  funds  so  that  future  products  can  share  a  common  FPGA  plat¬ 
form. 

•  program  versus  business  interest  needs.  Managers  recognize  the  need  for  AY  but  desire  to 
drive  costs  down.  More  AY  increases  costs  for  the  individual  program,  and  savings  from  re¬ 
use  of  previous  AY  ->  X  core  assets  is  often  not  recognized. 

•  organization.  B AE  Systems  must  recognize  design  versus  architecture  versus  product  line 
architecture  staff.  The  latter  is  a  small  number  with  a  chief  architect  across  programs.  The  or¬ 
ganization  also  includes  technical  leads  (called  technical  stewards)  across  eight  areas  (domain 
or  hardware  specialties).  Stewards  have  a  high  degree  of  awareness  of  cross-program  needs, 
consistent  with  the  product  line  vision.  A  concern  is  how  to  develop  chief  architect  skills. 

•  investment.  IRAD  investment  must  address  the  next  generation  architecture  and  look  down¬ 
stream  for  3-5  years. 

2.6.3  Recommendations 

Don  concluded  with  the  following  recommendations: 

•  Use  IRAD  funding  for  initial  platform  development. 

•  Strong  leadership,  both  technical  and  management,  is  required. 

•  Technical  leads  must  maintain  the  product  line  approach,  but  must  balance  between  not-too- 
little  or  too-much  effort  on  product  line  enhancements. 

•  Analyze  financial  costs  and  benefits — for  example,  SLOC  (source  lines  of  code)  saving, 
costs. 

•  Migrate  test  procedures  from  project- specific  assets  to  core  assets.  Use  IRAD  money  to  fund 
a  common  test  framework.  Do  continuous  integration  and  automating  of  builds. 

•  Individual  program  development  efforts  lead  to  branching  in  the  code  baseline  for  adaptation. 
Develop  configuration  management  procedures  to  merge  these  branches  back  into  the  base¬ 
line.  The  cost  of  merging  and  the  costs  of  baseline  adaptations  are  part  of  product  line  sus¬ 
tainment. 

•  A  technical  steward  should  manage  both  the  baseline  evolution  through  product  development 
and  the  need  for  rapid  turnaround  of  new  requests. 


2.7  The  RangeWare  Software  Product  Line  -  Ed  Dunn,  NUWC 

Ed  Dunn  of  NUWC,  Newport,  offered  a  brief  status  update  of  a  developing  product  line  for  un¬ 
dersea  test  and  training  ranges.  An  expanded  summary  of  Ed’s  remarks  is  available  to  support 
official  DoD  activities  through  email  to  Ed  at  edward.dunn@navy.mil. 
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3  A  Summary  of  the  Facilitated  Discussions 


Following  the  presentations,  the  group  participated  in  a  facilitated  discussion.  A  summary 
follows. 

3.1  Discussion  Topic:  To  achieve  product  line  success:  What  do  I  (as  a  supplier) 

want  in  a  program  office?  What  do  I  (as  an  acquisition  office)  want  in  a  supplier? 

Two  key  items  were  identified  for  the  supplier’s  “wish  list.”  Suppliers  would  like 

1 .  to  know  the  program’s  acquisition  strategy,  the  funding  available,  and  the  roadmap  for  the 
product  line  opportunity 

2.  to  have  a  Program  Office  that  is  steady  on  course  and  speaks  with  a  consistent  voice  from 
day  to  day 

It  was  recognized  that  the  latter  point  may  be  difficult  to  achieve  because  of  the  military  assign¬ 
ment  cycle.  This  cycle  sometimes  results  in  military  program  managers  who  do  not  take  a  strate¬ 
gic  view,  and  software  product  lines  are  a  strategic  effort.  A  change  in  leadership  may  also  mean  a 
change  in  strategy. 

When  considering  product  lines,  both  suppliers  and  program  offices  should  understand  that  a 
many-to-many  relationship  is  involved:  The  program  office  wants  multiple  contractors  to  foster 
competition;  suppliers  want  multiple  opportunities.  In  particular,  suppliers  who  have  created  a 
product  line  want  to  use  the  product  line  products  with  modification  on  multiple  systems  and  not 
be  restricted  by  the  data  rights. 

The  issue  of  competing  data  rights  was  a  primary  concern.  Depending  on  how  data  rights  are  ad¬ 
dressed,  they  could  limit  a  supplier’s  choice  of  even  bidding  on  a  product  line  opportunity.  Simply 
put,  suppliers  want  to  be  able  to  use  their  intellectual  property  elsewhere;  the  acquirer  wants  the 
data  rights.  As  one  participant  put  it,  “If  I  have  to  put  up  IRAD  money  I  need  to  be  able  to  recover 
it.  The  government  needs  to  provide  some  incentive  for  me  to  incur  the  IRAD  cost.  Also  if  the 
government  pays  for  the  product  line,  I  want  to  be  able  to  ensure  future  business — not  have  re¬ 
stricted  data  rights.”  There  has  to  be  a  business  case  that  shows  it  makes  sense  to  surrender  the 
intellectual  property. 

This  led  to  a  discussion  of  the  question,  “How  does  the  company  who  created  the  original  core 
assets  not  suffer  once  the  data  rights  are  given  to  other  suppliers?”  Some  suppliers  felt  that  their 
knowledge  of  the  application  domain  and  the  product  line  itself  would  provide  the  necessary 
competitive  advantage.  Others  felt  that  this  may  not  be  the  case,  saying  that  emulating  an  idea  is 
much  easier  than  coming  up  with  the  original. 

There  were  enough  loose  threads  in  this  discussion  for  several  people  to  request  a  special  breakout 
session  at  a  future  workshop  devoted  to  product  line  data  rights.  Setting  aside  the  data  rights  issue, 
the  discussion  led  to  the  next  topic. 
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3.2  Discussion  Topic:  What’s  in  it  for  the  supplier?  What  is  needed  to  incentivize 
suppliers? 

This  question  had  several  sub-parts: 

•  What  incentives  can  the  government  provide  to  encourage  suppliers  to  propose  a  product  line 
approach  even  if  it  is  not  a  hard  requirement? 

•  What  incentives  can  the  government  offer  to  encourage  suppliers  to  respond  to  a  product  line 
acquisition? 

•  What  are  the  barriers  from  a  supplier  perspective?  How  can  these  barriers  be  addressed? 

One  incentive  for  suppliers  would  be  the  existence  of  a  large  enough  market  for  derived  products 
to  recoup  the  investment.  If  the  government  is  funding  the  product  line  work  then  the  length  of  the 
contract  becomes  important. 

Several  suggestions  were  brought  up  about  promoting  product  lines.  One  was  for  ASSIP  to  spon¬ 
sor  workshops  for  government  (and  possibly  supplier)  contracting  officers  to  educate  these  acqui¬ 
sition  individuals  on  product  lines.  Other  educational  workshops  were  discussed  that  would  ex¬ 
plore  areas  such  as  product  line  cost  estimation,  product  line  metrics,  and  the  data  rights  issues, 
and  include  breakout  sessions  for  special  interests.  It  was  suggested  that  since  sy stems -of-sys terns 
are  likely  candidates  for  a  product  line  approach  this  might  be  an  incentive  to  have  more  suppliers 
participate  in  product  line  workshops.  Bob  Schwenk  said  that  ASSIP  will  support  workshops  with 
contractors  to  discuss  these  various  issues  (plus  FARS  and  DFARS).  Participants  urged  that  these 
workshops  should  involve  the  legal  people. 

Training  was  suggested  as  another  avenue  to  provide  a  push  on  the  government  side.  Many  or¬ 
ganizations  have  mandatory,  self-paced  training  programs.  Product  line  awareness  could  be  in¬ 
cluded  within  this  structure.  On  the  industrial-base  side,  while  some  schools  educate  students  in 
product  lines,  this  is  typically  at  the  graduate  level  and  is  not  widespread. 

Distributing  return  on  investment  data  throughout  the  Army  would  help  attract  attention.  There 
should  be  a  communications  strategy  targeting  different  media  that  reach  a  broad  audience,  in¬ 
cluding  senior  leaders. 

An  indirect  push  for  product  lines  is  the  Army’s  Software  Blocking.  Software  Blocking  mandates 
interoperability  and  backwards  compatibility.  Theater  needs  also  drive  program  managers  to 
move  towards  common  interfaces.  A  software  product  line  approach  may  provide  an  answer  for 
some  of  these  challenges. 

Contracting  strategies  to  promote  product  lines  were  discussed.  One  participant  suggested  that 
product  lines  could  be  encouraged  through  use  of  cost-plus  incentive  fees.  It  was  pointed  out  that 
use  of  an  award  fee  affects  near-term  behavior,  but  product  lines  are  about  the  long  term.  Program 
offices  are  starting  to  look  at  IDIQ-type  contracts.  Another  suggested  option  to  encourage  a  long¬ 
term  perspective  was  to  use  RDT&E  money  (versus  production  money).  In  the  words  of  one  par¬ 
ticipant,  “Most  contracting  officers  don’t  think  of  buying  software  in  the  same  way  as  they  think 
about  buying,  say,  tanks.  Why  not  long-range  funding  for  software  along  the  lines  of  contracts  for 
test  range  maintenance?” 
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As  in  any  lively  discussion,  interesting  points  are  made  that  don’t  strictly  follow  the  original  ques¬ 
tion.  These  points  included 

•  Technical  evaluation  criteria.  In  reference  to  what  to  include  in  sections  M  and  L  of  an  RFP 
(see  Section  2.2),  Bob  Schwenk  observed,  “There  has  to  be  a  relationship  with  our  contract¬ 
ing  offices;  we  don’t  do  a  good  job  of  spelling  out  the  criteria.”  When  considering  past  per¬ 
formance,  it  is  about  results,  not  just  following  a  checklist. 

•  Cost  estimation  and  metrics.  The  SEI  has  the  SIMPLE  Model  for  product  line  cost  estimation 
[Clements  2005,  SIMPLE  2009].  A  good  subject  for  a  future  workshop  would  be  product  line 
metrics. 
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4  Summary 


The  2009  Army  Product  Line  Practice  Workshop  explored  the  product  line  practices  of  organiza¬ 
tions  in  the  Army  community  through  sharing  of  experiences.  Thirty -five  people  attended,  repre¬ 
senting  22  organizations.  This  workshop  demonstrated  a  continuance  of  the  trend  revealed  during 
the  most  recent  workshops:  namely,  software  product  line  practice  is  becoming  a  reality  in  the 
Army  and  DoD.  All  the  presentations  were  based  on  experience  rather  than  plans  or  speculation. 
Benefits  are  being  quantified.  The  central  role  of  software  architecture  was  reaffirmed.  Challenges 
and  experience-based  solutions  were  discussed,  but  these  discussions  highlighted  the  increasing 
use  and  acceptance  of  software  product  line  practice. 

The  most  newsworthy  item  from  the  workshop  is  the  emergence  of  DoD  suppliers  that  have  suc¬ 
cessfully  addressed  the  oft-cited  barriers  of  business  justification  and  funding  for  a  product  line 
approach  for  government  business.  They  have  built  a  business  case  for  the  approach,  and  have 
been  able  to  establish  a  viable  strategy  for  funding  the  development  and  maintenance  of  a  core 
asset  base.  That  this  strategy  has  been  made  to  work  across  a  set  of  products  developed  for  differ¬ 
ent  customers  is  a  significant  achievement.  (Organizations  frequently  encounter  difficulties  when 
trying  to  move  from  a  customer- driven,  product- specific  funding  model  to  one  in  which  at  least 
some  of  the  funds  are  allocated  to  the  creation  and  maintenance  of  reusable  artifacts  that  can  be 
shared  across  several  product  development  efforts.  The  framework  practice  area  on  funding  a 
product  line  describes  the  challenges  and  provides  several  examples  of  funding  models  [Northrop 
2009b].) 

Participants  also  noted  other  appealing  aspects  of  today’s  product  line  successes. 

•  Given  the  right  leadership,  a  software  product  line  approach  can  grow  across  formerly  stove- 
piped  development  efforts. 

•  A  software  product  line  approach  allows  a  supplier  to  respond  quickly  to  an  RFP;  the  re¬ 
sponse  is  itself  a  core  asset. 

•  Licensing  the  software  product  line  can  be  part  of  a  viable  business  strategy. 

•  A  software  product  line  approach  should  be  considered  a  journey;  benefits  can  accrue  even  if 
the  first  attempt  isn’t  perfect. 

To  be  sure,  there  are  still  challenges,  as  evidenced  by  the  discussion  on  the  importance  of  the  data 
rights  issue  for  both  suppliers  and  acquirers.  But  the  underlying  attitude  was  that  product  lines  are 
real  and  appropriate;  now  people  want  to  know  how  to  buy  and  fund  them. 

The  post-workshop  critique  was  very  positive.  Participants  universally  agreed  on  the  value  of  the 
workshop  and  on  the  need  for  the  SEI  to  continue  work  in  software  product  lines  and  provide  re¬ 
lated  support  to  the  DoD.  The  consensus  among  long-time  attendees  was  that  this  workshop  bet¬ 
tered  the  1 1  previous  DoD  workshops,  both  in  terms  of  quality  of  presentations  and  participants 
and  in  the  progress  reported.  The  participants  recommended  that  the  AS  SIP  sponsor  follow-on 
workshops  this  year  on  specific  product  line  topics  and  another  general  workshop  next  year.  Bob 
Schwenk,  ASA  ALT,  was  equally  positive  and  agreed  with  the  recommendations. 
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If  you  have  any  comments  on  this  report  or  are  using  a  product  line  approach  in  the  development 
or  acquisition  of  software- intensive  systems  for  the  DoD  and  would  like  to  participate  in  a  future 
workshop,  please  send  email  to  Linda  Northrop  at  lmn@sei.cmu.edu. 
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